DeepLearningの本番環境にSageMakerを利用してる話

JAWS DAYS 2019でのランチセッションの発表です

1. DeepLearningͷຊ൪؀ڥʹ SageMakerΛར༻ͯ͠Δ࿩ ota42y (FiNC Technologies) JAWS DAYS 2019
2. ࠓ೔ͷ಺༰ • TensorFlowͷϞσϧͷຊ൪ӡ༻ͷ࿩ • SageMakerʹࡌͤସ͑ΔͱָͩΑ • एׯϋϚͬͨͷͰͦͷղܾํ๏
3. ࣗݾ঺հ • @ota42y • αʔόαΠυΤϯδχΞ • RubyͷਓͰ͕͢ࠓ೔͸΄΅PythonͷΈ • Amazon Web ServicesαʔόʔϨεϨγϐͬͯຊΛॻ͖·ͨ͠ (AWS Serverless Application Modelʹ͍ͭͯͷ࿩)
5. DeepLearningɺͯ͠·͔͢
6. FiNCΞϓϦͰDeepLearningͯ͠·͢
7. ৯ࣄͷه࿥ • ೔ʑͷ৯ࣄΛอଘ͢ΔػೳΛఏڙ • ඼໨Λొ࿥͢Δͱͦͷ೔ͷΧϩϦʔ΍PFCόϥϯε͕Θ͔Δ
8. ৯ࣄͷࣸਅղੳ • ৯ࣄͷࣸਅ͔Β඼໨ΛࣗಈͰೝࣝ͢ΔػೳΛఏڙ • ඼໨Λొ࿥ͤͣͱ΋؆୯ʹͦͷ৯ࣄͷӫཆ͕Θ͔Δ
9. ࣸਅͷೝࣝ͸Faster R-CNN • Կ͕ࣸͬͯΔ͔൑ఆ͢ΔλεΫ(Object Detection) • TensorFlowͰֶशͤͨ͞Faster R-CNNϞσϧ
10. RubyͷձࣾͳͷͰϞσϧͷӡ༻͕೉͍͠ • FiNCͷαʔό͸΄ͱΜͲ͕Ruby on Rails • ৯ࣄͷࣸਅΛอଘ͢Δαʔό΋Ruby on RailsͰͰ͖͍ͯΔ • ͦͷ··Ͱ͸TensorFlowͷϞσϧ͸࢖͑ͳ͍
11. RubyͱTensorFlowΛͲ͏૊Έ߹ΘͤΔ͔ • Ruby͔ΒPyCallͰPythonΛݺͿ • GPUαʔόʹPythonͱRubyͷ྆ํͷ؀ڥΛ΋ͨͤΔͷେม • PythonͷTensorFlowΛॲཧ͢ΔΞϓϦαʔόΛཱͯΔ • ϝϯς͕େม • TensorFlow ServingΛ࢖͏ • ͜Ε͕ྑͦ͞͏
12. TensorFlow ServingͰAPI࿈ܞ • TensorFlowͷϞσϧΛӡ༻͢ΔެࣜαʔόΞϓϦ • ෳ਺Ϟσϧͷ؅ཧɾόʔδϣχϯάՄೳ • ֤Ϟσϧʹରͯ͠REST/gRPCͷAPI͕ఏڙ͞ΕΔ • αʔόΞϓϦέʔγϣϯͷίʔυΛॻ͔ͳͯ͘ྑ͍ • Docker image΋͋ΔͷͰ؆୯ʹཱͯΒΕΔ
13. ӡ༻͸ΘΓͱେม… • ϞσϧΛӡ༻͢ΔαʔόΛॻ͔ͳ͍͍ͯ͘ͷ͸ศར • ΞϓϦαʔόΛ࡞Δ͜ͱʹूதͰ͖Δ • GPUαʔόͷ؅ཧɺεέʔϧ͸ࣗ෼ͨͪͰ΍Δඞཁ͕͋Δ • ϞσϧϑΝΠϧͷߋ৽ϑϩʔߏங • DockerԽͯ͠ΔͱΑΓ΍΍͘͜͠… • gRPCͱALBͷ૬ੑ͕ѱͯ͘…
14. ͦ͜Ͱ AWS SageMaker ࠓ5෼͙Β͍ͳΒ΋͏গ͠ৄ͘͠࿩͍͖ͯ͠·͢
15. AWS SageMakerͱ͸ • ϑϧϚωʔδυͳػցֶशαʔϏε • TensorFlow౳ͷ؀ڥ͕ߏங͞ΕͨJupyter Notebook • ֶश༻GPUαʔό΍αʔόΛ؅ཧ͢ΔSDK(෼ࢄֶश΋ରԠ) • ֶशͨ͠ϞσϧΛӡ༻͢Δproductionαʔό
16. SageMakerͩͱ Ϟσϧͷ։ൃɾֶशɾӡ༻ શ෦Ͱ͖Δ
17. طଘϞσϧͷӡ༻΋Ͱ͖Δ • ΋ͪΖΜ։ൃɾֶशɾӡ༻ͷҰ෦͚ͩ࢖͏͜ͱ΋Ͱ͖Δ • ࠓճ͸ࣗࣾGPUαʔόͰ࡞ͬͨTensorFlowͷϞσϧΛɹɹɹɹ SageMakerͰӡ༻ • ӡ༻͚ͩར༻Ͱ΋σϓϩΠ͕؆୯ʹͳΓɺ
 GPUαʔόͷ؅ཧ͔Βղ์͞ΕΔ
18. جຊతͳػೳ΋੔ͬͯΔ • Φʔτεέʔϧػೳ΋ఏڙͯ͠Δ • ৽͍͠Ϟσϧ͸ΠϛϡʔλϒϧσϓϩΠϝϯτ͞ΕΔ • ಛʹઃఆͱ͔͸ෆཁ • deploy͢Δ͚ͩͰ৽͍͠ͷ্ཱ͕͕͔ͪͬͯΒަ׵ • ෳ਺ϞσϧͷॏΈ෇͖ӡ༻΋Ͱ͖Δ
19. TensorFlowͷϞσϧΛಈ͔͢ • (࣮࣭)ϑϧϚωʔδυͳTensorFlow Serving͕࢖͑Δ • ϞσϧΛ༻ҙ͢Ε͹Φʔτεέʔϧ͢ΔREST API͕ੜ͑Δ • S3ʹϞσϧϑΝΠϧΛஔ͍࣮࣭ͯ3ߦॻ͚ͩ͘ from sagemaker.tensorflow.serving import Model model = Model(model_data='s3://xxx/model.tar.gz', role=……) model.deploy(initial_instance_count=1, instance_type=‘ml.c5.xlarge')
20. Ruby͔ΒݺͿ • AWSͷSDKʹJSONͰσʔλΛ౉͚ͩ͢ client = Aws::SageMakerRuntime::Client.new(access_key_id: ....) res = client.invoke_endpoint({ endpoint_name: "endpoint", body: data.to_json, content_type: ...})
21. Before After S3 ECS্ͷ Ruby on Rails ECS্ͷ TensorFlow Serving ࣸਅΛS3͔Βऔಘ ࣸਅΛ൑ผͤ͞Δ (gPRC) S3 ECS্ͷ Ruby on Rails SageMaker ࣸਅΛS3͔Βऔಘ ࣸਅΛ൑ผͤ͞Δ (JSON)
22. ௒؆୯
23. Ͱ͸ͳ͔ͬͨ
24. • S3ʹอଘͨࣸ͠ਅΛαʔό͕ಡΈࠐΈɺTensorFlowʹૹΔ • SageMakerʹૹΔσʔλ͸5MB·Ͱͷ੍ݶ͕͋Δ • ը૾ͷ୹ล͕700ʙ800pxͰ5MBҎ্ʹͳͬͯࢮ๢ • ʘ(^o^)ʗ SageMakerͷ੍ݶʹҾ͔͔ͬͬͯ͠·ͬͨ
25. • ݩʑ͸ECS্ͷTensorFlow ServingʹgRPCͰૹͬͯͨ • ըૉͷ഑ྻ͸JSONʹ͢Δͱ༰ྔ͕๲ΕΔ • (n*n) pxͷRGB஋͔ΒͳΔ഑ྻΛJSONʹ͢Δͱ໿2ʙ3ഒʹ
 ؒͷεϖʔεແ͠([1,2] rubyͱ͔) ؒͷεϖʔε͋Γ([1, 2] pythonͱ͔) • ࠓ·Ͱ͸໰୊ͳ͔͚ͬͨͲJSONͩͱ…ʘ(^o^)ʗ Կ͕໰୊ʹͳ͔ͬͨ 8n2 + 2n + 1 3n2 11n2 + 2n 3n2
26. S3 ECS্ͷ Ruby on Rails SageMaker ࣸਅΛS3͔Βऔಘ ࣸਅΛ൑ผͤ͞Δ (JSON) ͕͜͜༰ྔΦʔόʔ
27. • TensorFlow Serving͕ड͚Δͷ͸σʔλͷ഑ྻ • શ෦ࠐΈͰͷΞϓϦͳͷͰ೚ҙͷॲཧΛೖΕΒΕΔ෦෼͸ແ͍ • ͱ͍͏͔ެࣜυΩϡϝϯτʹॻ͍ͯ͋Δ • ଧͭखͳ͔͠…ʘ(^o^)ʗ TensorFlow Servingతʹແཧͦ͏ 5FOTPS'MPX
    4FSWJOHCBTFE
    TFSWFS
    UP
    QSPWJEF
    B
    TVQFSTFU
    PG
    UIF
    5FOTPS'MPX
    4FSWJOH
    3&45
    "1*
    5IJT
    PQUJPO
    EPFT
    OPU
    SFRVJSF
    PS
    BMMPX
    B
    DVTUPN
    QZUIPO
    TDSJQU
28. ࠓ11෼͙Β͍ͳΒ΋͏গ͠ৄ͘͠࿩͍͖ͯ͠·͢
30. SageMaker্Ͱ΋લॲཧΛ࣮ߦͰ͖Δ • TensorFlow ServingͰ͸ͳ͍σϓϩΠ΋αϙʔτͯ͠Δ
 ɹ(Python-based Endpoints) • ͦͷ৔߹͸લॲཧɾޙॲཧΛ࣮ߦͰ͖Δ
31. SageMaker্Ͱ΋લॲཧΛ࣮ߦͰ͖Δ • σϓϩΠ࣌ʹ࣮ߦ͢ΔϑΝΠϧΛؚΊΔ͚ͩ
 (class͕ҧ͏ͷͰ஫ҙ) • requirements.txtͰґଘύοέʔδ΋ೖΕΒΕΔ from sagemaker.tensorflow import TensorFlowModel model = TensorFlowModel(model_data = 's3://xxx/model.tar.gz', ɹɹɹɹɹɹɹɹɹɹɹɹɹɹ entry_point='pred.py', role=...) model.deploy(initial_instance_count=1, instance_type='ml.c5.xlarge')
32. entry_point.pyͷྫ import numpy as np def input_fn(serialized_input, content_type): """An input_fn that loads a pickled object""" if request_content_type == "application/python-pickle": deserialized_input = pickle.loads(serialized_input) return deserialized_input else: # Handle other content-types here or raise an Exception # if the content type is not supported. pass def output_fn(prediction_result, accepts): """An output_fn that dumps a pickled object as response""" if request_content_type == "application/python-pickle": return np.dumps(prediction_result) else: # Handle other content-types here or raise an Exception # if the content type is not supported. pass
33. εΫϦϓτ͸͜Μͳײ͡ import boto3 import json import numpy as np from PIL import Image import io AWS_S3_BUCKET_NAME = 'NAME' s3 = boto3.resource('s3') def input_fn(serialized_input, content_type): request_json = json.loads(serialized_input.decode()) s3_key = request_json['key'] response = s3.Object(AWS_S3_BUCKET_NAME, s3_key).get() image_bytes = response['Body'].read() img = Image.open(io.BytesIO(image_bytes)) bmp_data = np.asarray(img) return bmp_data[np.newaxis, :, :] ೖྗ͸JSONͰ
 {“key”: “/path/to/a.jpg”}
 S3͔Βը૾ΛऔΓ
 RGBʹม׵ 
 ͜͜ʹ༰ྔ੍ݶ͸ͳ͍
34. Before After S3 ECS্ͷ Ruby on Rails SageMaker ࣸਅΛ൑ผͤ͞Δ (JSON) S3 ECS্ͷ Ruby on Rails SageMaker ࣸਅΛS3͔Βऔಘ S3ͷkey͚ͩૹΔ (༰ྔখ) ࣸਅΛS3͔Βऔಘ ͕͜͜໰୊ͩͬͨ
35. Before After S3 ECS্ͷ Ruby on Rails SageMaker ࣸਅΛ൑ผͤ͞Δ (JSON) S3 ECS্ͷ Ruby on Rails SageMaker ࣸਅΛS3͔Βऔಘ S3ͷkey͚ͩૹΔ (༰ྔখ) ࣸਅΛS3͔Βऔಘ ௨৴ྔ͕ݮͬͯϨεϙϯε΋ૣ͘
36. ༏উ
37. Ώ͘Ώ͘͸։ൃ΍ֶश΋SageMakerʹ • AIͷݚڀऀʹ։ൃɾֶश༻αʔόͷӡ༻Λߟ͑ͤͨ͘͞ͳ͍ • ṷ͸ṷ԰ɺݚڀऀʹ͸ݚڀऀʹ͔͠Ͱ͖ͳ͍͜ͱΛ • શ෦ΤϯδχΞ͕ೖΔͷ΋ίϛϡχέʔγϣϯͷແବ͕…ɹɹɹ • SageMaker͸αΫοͱͰ͖ΔͷͰɺ͜Ε͙Β͍ͳΒ͍͚ͦ͏
38. ·ͱΊ • AWS SageMaker͸։ൃɾֶशɾӡ༻શ෦ग़དྷΔ • Ϟσϧͷӡ༻͚ͩར༻͢Δͷ΋͘͢͝ศར • લॲཧɾޙॲཧ͕Ͱ͖ΔͷͰ΄ͱΜͲͷ༻్Ͱ࢖͑Δ • SageMakerͰ৭ʑ͍ͨ͠ਓ͸we’re hiringͯ͠·͢(
    ƅ㱼ƅ)oኯʄ
39. TensorFlow, the TensorFlow logo and any related marks are trademarks of Google Inc. TensorFlow Logo Copyright © 2019 The TensorFlow authors Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0. Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.